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Abstract 

In this work we address the interpretation of neuronal activity as performing pattern recognition. After briefly 
describing biological neuronal dynamics, we present a simplified model of a neuronal cell involving linear combination 
of inputs followed by a hard-limiting non-linearity. Then, two possible interpretations of neurons are discussed: as 
linear discriminators, and as template matchers. The concept of linear pattern morphing is described as a preparation 
for addressing a type of artificial neuronal network for non-supervised learning known as self-organized maps (SOMs), 
which is then presented and illustrated. 


“Mientras el cerebro sea un misterio, el uni verso continuara 
siendo un misterio.” 

Santiago Ramon y Cajal. 

1 Introduction 

Unicellular organisms... multicellular organisms... neu¬ 
ronal networks... science! 

As with cosmic matter, cells - the atoms of biology - 
presents a tendency to come together, giving rise to organ¬ 
isms capable of more flexible and intricate behavior. The 
neuronal networks underlying the nervous system repre¬ 
sent a further step along this trend. By possessing pro¬ 
longations (dendrites and axons), neuronal cells became 
specialized in making connections, necessary for transmit¬ 
ting and processing information. The thus obtained in¬ 
formation processing systems, capable of impressive per¬ 
formance, ultimately gave rise to human society, culture 
and science. Even though not often realized, much of 
these activities actually corresponds to instances of pat¬ 
tern recognition , informally the task of assigning new or 
previously defined categories to world entities. The very 
reading of these lines provides a related example, as we 
associate meaning to each word and sentence. 

The reference performance of biological neuronal net¬ 
works have continuously motivated efforts from the scien¬ 
tific community in developing artificial neuronal networks 
having as main objectives: (i) to model (e.g. pQ) and bet¬ 


ter understand biological neuronal networks; and (ii) to 
develop systems that can be applied in several pattern 
recognition and artificial intelligence tasks. 

The present work is aimed at presenting and discussing 
neuronal cells in one of their most important roles: pat¬ 
tern recognition. Here, after briefly presenting the biolog¬ 
ical operation of neurons, we describe a simple respective 
mathematical model that allows us to address two possi¬ 
ble respective interpretations of neuronal action as pat¬ 
tern recognition: neurons as linear discriminators, and 
neurons as template matchers. 

Next, we introduce the concept of pattern linear mor¬ 
phing, corresponding to interpolations between two given 
patterns. These principles are then adopted to describe 
and illustrate an interesting artificial neuronal network 
model, namely self-organizing maps - SOMs. In these 
networks, which are primarily intended for non-supervised 
patter recognition, each linear neuron in an artificial ‘cor¬ 
tex ; acts as a pattern recognizer, receiving information 
from the whole pattern being presented, changing not 
only its weights, but also the weights of other nearby neu¬ 
rons, so as to become more similar to the present input. 
This dynamics, which incorporates interesting biological 
principles such as topographic cortical organization and 
function similarity between nearby cells, can result in self- 
organized maps, with regions associated to the main types 
of presented patterns. Interestingly, trajectories along the 
‘cortex’ can often be understood as morphings between 
the identified categories of patterns. 
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By corresponding to a type of convolutional artifi¬ 
cial neuronal networks, SOMs are closely related to deep 
learning (e.g. 0 ) , therefore providing an interesting entry 
point to this impressive area. 

2 A Brief Review of Neuronal Dy¬ 
namics 

Neurons can be thought of as biological cells special¬ 
ized for information transmission and processing. Indeed, 
while many biological cells have nearly round shapes, neu¬ 
rons are characterized by prolongations (axons and den¬ 
drites) allowing them to interconnect as networks. Being 
themselves cells, much can be known about neurons by 
studying respective biological principles (e.g. 0 ). 

A cell is a biological entity delimited by a respective 
membrane (bilayer) composed of phospholipids, which has 
high electrical impedance and blocks most matter flow ex¬ 
cept for some small ions. Cells can exchange information, 
mass and energy through proteins that goes through the 
cell membrane and act as transmembrane channels. These 
channels can open and close under control of electric po¬ 
tential, other molecules, etc. Some transmembrane pro¬ 
teins act as ion pumps , transferring ions in or out the cell 
interior. Such pumping requires energy consumption. 

As a consequence of charge and osmotic balance 
through the membrane, a negative electric rest (or equi¬ 
librium) potential of approximately —60 mV is established 
inside the cell with respect to its exterior. Any increase 
of this voltage is typically understood as a depolarization. 

As cells, neurons incorporate all the above structures 
and mechanisms. In addition to that, neurons have two 
main types of tree-like extensions: (i) dendrites , which 
receive information; and (ii) axon , which transmit infor¬ 
mation to other cells. The diagram in Figure [l] illustrates 
a highly simplified neuronal cell, including cell soma (or 
perykarion), dendrites, axon, and synapses. 

The connection between two neuronal cells is accom¬ 
plished through synapses , incorporating the pre-synaptic 
button (at the axons extremities), the synaptic cleft (the 
synaptic interstice between the two cells), and the post- 
synaptic cell membrane. The action of a synapsis is 
mainly controlled by the action potential it can receive 
from the respective axon. The action potential is char¬ 
acterized by spikes or impulses of electric depolarization 

Once a synapsis start receiving spikes, it releases neu¬ 
rotransmitters into the cleft, which act on the postsynap- 
tic transmembrane channels, allowing intake of ions such 
as Na + . Being positive, these ions depolarize the postsy- 
naptic region close to the respective synapsis, and this de¬ 
polarization then propagates through diffusion along the 



Figure 1: The basic components of a neuronal cell include its cell 
body (soma) and respective nucleons , dendrites , axon and synapses. 
A synaptic connection established by a given cell define a reference 
for the pre-synaptic neuron (the same cell) and the post-synaptic 
neuron (the cell that is being contacted). Input signals are received 
at the dendrites, and the neuronal output corresponds to electric 
pulses, known as action potentials , that propagate along the axon 
until the respective synapsic terminations. 


dendrites and soma. The depolarizations emanating from 
several synapses are combined, and when the electric po¬ 
tential at the post-synaptic implantation cone increases 
by approximately 20 mV (typically), voltage-controlled 
transmembrane channels open, allowing intake of Na + , 
which further depolarizes the implantation cone and be¬ 
ginning of the axon. This implies an abrupt change of the 
local cell potential, giving rise to an action potential. 

Depolarization can be transmitted through the axon 
internal as well as externally, inducing a chain reaction 
along the axon, corresponding to an action potential. As 
the action potential ceases locally, ion pumps on the im¬ 
plantation cone and axon act in order to release the excess 
Na + , which is done at the expense of energy consumption. 
The presence of myelin sheaths along the axons of sev¬ 
eral types of neurons can act as an insulator, promoting 
saltatory conduction through voltage fields external to the 
axon, minimizing energy consumption, as only the pumps 
between sheaths (nodes of Janvier) need to operate, also 
implying faster action potential transmission. 

The effect of a pre-synaptic action potential on the sub¬ 
sequent post-synaptic neuron depends on the synaptic ef¬ 
ficiency, in the sense that the higher this efficiency the 
higher the respectively implied depolarization. Synaptic 
efficiency is largely influenced by the previous history of 
joint activation of the two involved cells, mediated by neu¬ 
romodulators. Observe that a great extent of biological 
memory corresponds to the respective synaptic weights, 
managed by neuromodulators. 

This is only an approximate and generic description of 
the neuronal dynamics, which involves many other details 
and can vary in several manners and cases. 
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3 A Simple Neuronal Model urega). 


The above described principle of neuronal action can be 
modeled at several levels of complexity. Figure [2] depicts 
a relatively simple, mesoscale mathematical model of a 
neuron. 



Figure 2: A simple model of the biological action of a neuron as 
revised in Section [2] The linear combination of the depolarizations 
implied by active synapses into the implantation cone is modeled 
in terms of the inner product = YliLi XiWi, which is 

then fed into a nonlinear hard-limiting function A(), which models 
the abrupt generation of action potentials in the implantation cone, 
yielding the respective output P(X i, X 2 ,. .., Xjv)- 


The dendrites are represented by input connections as¬ 
sociated to each of the input values JQ, i = 1,2,. ..,7V, 
which can be understood as the average action potential 
excitation received from incoming axons. These intensi¬ 
ties are multiplied by respective weights Wi ,which cor¬ 
respond to the respective synaptic efficiencies. The de¬ 
polarizing effect of these input is linearly integrated, im¬ 
plementing the respective inner product S = = 

XiWi, modeling the the depolarization of the im¬ 
plantation cone as a consequence of the combined effect 
of diffusion of the depolarizations at the synapses. 

The integrated activation S is then fed to a hard- 
limiting function A(), modeling the threshold operation 
at the implantation cone. A sigmoid function such as 
that in Equation [l] can be used for this finality. 

p == irb? (1) 

where /3 is a parameter that controls the smoothness of 
the sigmoid transition, becoming sharper for larger values 
of (3. Observe that there are other types of sigmoids and 
activation functions that can be adopted. It is possible 
to consider a simplification of this model in which the 
non-linear part is omitted, yielding a respective a linear 
neuron. 


4 The Neuron as a Linear Dis¬ 
criminator 

One possible interpretation of neuronal action is as lin¬ 
ear discrimination. Consider the situation shown in Fig- 



P(x.y) 



(b) 


Figure 3: The action of a two input neuron as a linear discriminator: 
any point (x, y) to the lefthand side of the separatrix (red line) will 
yield positive sign, while those to the righthand side will be negative. 


This single neuron has three inputs: x, y and a constant 
signal c. Therefore, it will produce as result 

P(x, y) = A(xw x + yw v + c ) (2) 

It is interesting to find the set of points (x, y ) that 
makes xw x + yw y + c equal to zero: 

_ C + XW x . 

0 = xw x + yw y c y = -(3) 

Wy 

Observe that the above result corresponds to a line 
equation, defining a respective line in the (x,y) space as 
illustrated in red in Figure [3] 

Figure [3]^b) depicts the input space (x, y) as well as the 
sign of P(x, y) obtained in this space. It can be verified 
that the neuron can identify, through the sign of P(x,y ), 
in which of the two half-planes any instance (x, y) of the 
signal is. For instance, any input above the separating 
line L will yield + sign, and any input below that line 
will imply — sign. 

The above characterized action of neurons as linear dis¬ 
crimination can immediately be applied to pattern recog¬ 
nition. For instance, in the case in Figure [3j each of the 
+ and — regions can be associated to two different cate¬ 
gories, so that a new point mapped at (x,y) will be as¬ 
signed the respective sign as its recognized category. 
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5 The Neuron as a Template 
Matcher 

Let u = [u x . u y ] T and p = [v x , v y ] T be two vectors in 'R 2 . 
Their inner (or scalar) product can be defined as the fol¬ 
lowing scalar value: 

(u, v) = U x v x + UyVy = \u\\v\cos(6) (4) 

where 0 is the smallest angle between these two vectors. 
Figure [4] illustrates the two vectors u and v. 



Figure 4: Two vectors u and v and the smallest angle 6 be¬ 
tween them. The respective inner product (u, v) can be understood 
as the magnitude of the projection of one vector into the other 
(e.g. \u\cos(6)) multiplied by the latter magnitude (e.g. \v\). 

The magnitude of the projection of u into v can be im¬ 
mediately obtained by applying trigonometry as \u\cos(0) 7 
implying that the vector product between those two vec¬ 
tors corresponds to the the value obtained by multiplying 
the magnitude of the projection of one vector by the mag¬ 
nitude of the other vector. 

Therefore, if the magnitudes \u\ and \v\ are both kept 
constant, the inner product will achieve its maximum 
value when the projection of u into v (or, alternatively, 
v into u) will be maximum, which is achieved when both 
vectors are parallel and 6 = 0. 

When the magnitude of the projection of one vector 
into the other becomes null (0 = 7r/2), the respective 
inner product equals zero and the vectors are said to be 
orthogonal. However, this is not the minimum inner prod¬ 
uct value, which is rather achieved when the two vectors 
are colinear but in opposite directions (6 = 7r). 

A particularly interesting interpretation of the inner 
product between two vectors is as a measurement of the 
similarity between those two vectors. The similarity will 
be highest when the projection is maximum (0 = 0), and 
lowest when the vectors are opposed (0 = n). 

The above interpretation hold also when the vectors 
are in higher dimensional spaces. In particular, the un¬ 
derstanding of the inner product of two vectors as a mea¬ 
surement of similarity provides one of the keys for better 
understanding neuronal networks in a more conceptual 
manner. This can be immediately achieved by taking the 
synaptic input signal as one of the vectors (e.g. u) and 
the set of synaptic weights as another vector (e.g. v). 


Figure |5] illustrates the action of a simple neuron as a 
template matcher. 


input pattern 



Figure 5: A simple neuron, with 5 inputs, acting as a template 
matcher by taking the inner product between the input values and 
respective weights, shown in gray levels. A relatively high output P 
is expected in case the signal resembles the signal, as in the present 
case. 

Let’s first consider a linear neuron, i.e. without the hard 
limiting function. The respective output corresponds pre¬ 
cisely to the inner product between the input signal and 
the synaptic weights. Thus, the neuron will exhibit strong 
output whenever the input signal resembles its respec¬ 
tively associated weights. When the hard limiting non¬ 
linearity is included, the neuron will be active only when 
this similarity exceeds the respective threshold. 

In a sense, we can say that any of these two types of 
neurons perform template matching (a kind of pattern 
recognition), yielding high output whenever the signal is 
recognized as correspondingly closely to its weights. The 
conceptual potential of this interpretation will be illus¬ 
trated when we discuss self-organizing maps in Section [7J 

However, this approach has an intrinsic shortcoming 
in the sense that the inner product depends strongly on 
both the magnitudes of the two involved vectors. In case 
u represents the synaptic weights, and therefore has fixed 
magnitude, a signal very distinct from the weights, but 
with large magnitude, can still induce a strong (false) in¬ 
dication of matching. 

The above discussed template matching operation of a 
neuron is the basis for convolutional neuronal networks 
(e.g. 0 ). Indeed, this follows from the fact that the con¬ 
volution between two signals (e.g. @ 1 ) can be understood 
as performing inner products between one of the signals 
and a domain-reversed version of the other signal. 

In the next section we will discuss this problem in more 
detail and also present a possible simple and effective al¬ 
ternative scheme. 
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6 Interpolating Between Patterns 

Given two patterns u and v, it is possible to interpolate 
patterns between them, an operation that is often called 
morphing of one pattern into the other. Interestingly, lin¬ 
ear morphing is not only interesting and with application 
potentials, but it is also simple to be implemented by us¬ 
ing the following expression: 

f(a) = u-\-a(v — u) (5) 

allowing vector u to be progressively morphed, accord¬ 
ing to increasing values of a G [0,1], into the other given 
vector v. Figure [6] illustrates this basic principle of linear 
morphing from a vector u into another vector v. 



Figure 6: Given two vectors u and v , it is possible to obtain linear 
interpolations from u to v by obtaining the vector difference v — u 
and defining the interpolated vectors as r(a) = u + ctv for some 
a G [0,1]. 

An example of pattern morphing is presented in Fig¬ 
ure [7| where the first pattern (top, left) is progressively 
morphed into the last pattern (bottom, right). 


□ 

□ 

□ 

□ 

0 

0 

❖ 

0 
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Figure 7: Stages along the linear morphing of the first pattern (top, 
left) into the last pattern (down, right) by using Eq. [ 5 ] 


7 Self-Organizing Maps — SOMs 

Self-organized maps, SOMs for short, are artificial neu¬ 
ronal networks (e.g. 0 ) developed from the 70’s, espe¬ 


cially by T. Kohonen (e-g. @)> as artificial neuronal net¬ 
works for non-supervised classification (clustering). These 
networks are aimed at an increased neurobiological re¬ 
alism in the sense that the neurons are spatially orga¬ 
nized along a typically 2D cortex and that each neuron 
influences other neurons in its neighborhood so that their 
properties become more similar, a property often observed 
among neurons in biological cortices (e.g. 0 ). 

Figure [8] shows the overall structure of one a simple 
SOM network. 



Figure 8: The overall organization of a SOM network. An N x M 
input pattern is presented to an Q x R neuronal network (‘cortex’). 
Each linear neuron [k,l] receives input from all components [i,j] 
of the input pattern, through respective weights ^[i,j][k,l\i yielding 
respective output Cfy.^] corresponding to the inner product between 
the presented pattern and the respective weight, therefore providing 
an indication of the similarity between these two vectors. 

The network itself consists of a ‘cortex’ containing QxR 
linear neurons, which are typically organized as a lattice 
[fc, /], i = 1, 2,..., Q; j = 1,2,...,#. Interestingly, there 
is no interconnection between the neurons in the ‘cortex’, 
as their interaction is implemented during the learning 
stage. In this configuration, the patterns correspond to 
N x M matrices (other types of patterns are also pos¬ 
sible), each pixel being fed into all neurons in the 

‘cortex’, scaled by respective weight [&,/]• Therefore, 
the output of each neuron can be thought as correspond¬ 
ing to the inner product between the pattern and the 
respective weight, yielding an indication of the similarity 
between these vectors (observe that matrices and images 
can be transformed into respective vectors by row or col¬ 
umn stacking). 

Though aimed at non-supervised pattern recognition, 
typically a SOM will require training and subsequent rec¬ 
ognizing stages. In the training stage, patterns are pre¬ 
sented to the network, which updates its weights accord¬ 
ing to the learning rule illustrated in Figure [9j 

Basically, for each neuron [fc, /], the vector difference be- 
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Figure 9: The learning rule in a typical SOM. Given a current 
pattern X being currently presented, a neuron [k, l] can be biased 
towards recognizing it by making its respective weights ^[k,i] more 
similar to the pattern. Mathematically, this operation is analogous 
to the morphing of the weight vector into the input pattern (see 
Fig-0, yielding a modified weight W[ k ,i] = W[ k ,i] + cl(X - W[ k ,i]), 
where a is a learning rate belonging to the interval [0,1]. The higher 
the value of a, the more similar the new weight will be to the input 
pattern X. 


tween the input pattern and the respective weight [W^j, 
yielding (X — W[ k ,i\), is obtained and then multiplied by 
the learning rate a and added to [VF^/], with a E [ 0 , 1 ]. 
Observe that this operation makes the weight of the con¬ 
sidered neuron more similar to the input pattern. Indeed, 
in case a were 1 , the weight would become identical to 
the input pattern. Smaller values of a are often used in 
order to cater for robustness and generalization during 
the pattern recognition. 

Initially, all weights of a SOM network are initialized 
with uniformly random values in the range [0,1]. Once an 
input pattern X has been applied to the network, it is op¬ 
erated by each neuron in the ‘cortex’ (i.e. the inner prod¬ 
uct between the input signal and weight is calculated for 
every neuron), and the neuron yielding the maximum out¬ 
put is identified. Observe that this neuron corresponds to 
that with weights most similar to the input pattern. The 
above learning rule is then applied not only to the max¬ 
imum output neuron, to further reinforce its similarity 
with the input pattern, but also to its neighbors within 
a circle of radius R along the ‘cortex’, as illustrated in 
Figure [lOj 

The learning rate a is often made to vary with the 
distance r and number of training steps t (e.g. number 
of patterns trained so far). Generally, a can be made to 
decrease exponentially with r, implementing the tendency 
of neurons further away from the maximum output neuron 
to be less affected than those that are closer to that cell. 
Also, a is often made to decrease along the training steps 
£, in order to contribute to the stability of the unfolding 
weight modifications. A possible means to achive this is 
by expressing a as: 
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Figure 10: (a) Given an input pattern X presented to a SOM during 
the learning stage, all neurons within at distance up to R from the 
maximum output cell along the cortex have their weights modified 
in order to become more similar to the input signal. 


a = a(r, t) = 7 exp < — 




Nt J (6) 

where c is a constant (e.g. 0.5) controlling the inten¬ 
sity of the exponential decay, 7 is a constant defining the 
maximum learning rate, and N t is the total number of 
patterns to be trained. Figure 11 illustrates a(r, t) in 


terms of r and t for a specific parameter configuration. 

Observe that, though the SOM neurons are not directly 
interconnected, they nevertheless influence one another 
during the learning stage as a consequence of the above 
described modifications not only of the maximum output 
cell, but also of its respective neighbors. 

The basic steps of the described learning stage of a SOM 
network are summarized in the pseudo-algorithm [l] 


Algorithm 1 SOM 


1. initialize the SOM network weight with values uni¬ 
formly distributed in the interval [ 0 , 1 ]. 

2. for each pattern X to be trained: 

(a) input X into the SOM; 

(b) find cortical cell C max with the maximum out¬ 
put; 

(c) apply the learning rule to the weights of Cmax 
and its neighbors; 

3. Output weights. 


Once a SOM network has been trained, it will tend to 
present adjacent regions associated to each of the types 
of trained patterns. Neurons inside each of these regions 
will tend to present high output when signals resembling 
the associated pattern is fed into the network. A possible 
means to identify the patterns to each obtained region is 
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Figure 11: The learning rate values considered in terms of the dis¬ 
tance r from the maximum output cell, which are progressively re¬ 
duced along the training steps t. In this figure, we adopted R = 7, 
c = 0.5 and 7 = 0.2. 


by comparing (e.g. by taking the Euclidean distance) the 
respective weights of each neuron with the patterns and 
taking the pattern that gives the smallest distance. 

Now, we present a complete example of training a SOM, 
taking into account the three basic patterns shown in Fig¬ 
ure [l2] (a-c). Scrambled versions of these patterns, ob¬ 
tained by shifting the pixels, as illustrated in Figure [l2]^d- 
f), are also used during the training stage. A total of 2000 
such input patterns were fed into the SOM network during 
its training. 

The resulting weights are assigned to patterns by con¬ 
sidering their minimal Euclidian distance. Figure [T 3 ] ( cen¬ 
ter) depicts the resulting ‘cortex’, already associated to 
each of the three considered types of patterns. Observe 
that the cortical space is partitioned among three regions 
corresponding to each of the types of patterns. A possible 
trajectory along the ‘cortex’ is indicated in cyan, perform¬ 
ing a loop while going through each of the three obtained 
regions, and some of the respective patterns (defined by 
the associated weights) found along this trajectory are 
also shown around the ‘cortex’ figure. 
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Figure 12: The three reference patterns (a-c) and respective exam¬ 
ples of scrambled variations (d-f) used during the described SOM 
example. 


Interestingly, the patterns found along such trajectories 
tend to vary smooth and progressively, in a way similar to 
pattern morphing. This allows generalization of the SOM 
to several patterns that have not been trained, as well 
as caters for some level of robustness derived from some 
level of redundancy in the achieved patter representation. 
For instance, in case some of the neurons fail (e.g. in a 
respectively modeled biological network), the neighboring 
cells would still be able to provide some level of pattern 
identification. 

Once the SOM is trained and associated to the respec¬ 
tive pattern types, new patterns can be fed into it, being 
recognized as a possible instance of the type of pattern 
corresponding to the neuron yielding the maximum out¬ 
put. 


8 Concluding Remarks 

Neurons, the ‘atoms’ of the nervous system, can often 
be understood as performing pattern recognition. In 
this work, we discussed two possible such interpretations 
as: (a) linear discrimination, and (b) template match¬ 
ing. We have seen that the output of neurons as in the 
adopted mathematic-computational model perform the 
inner product between the input pattern and their re¬ 
spective weights, therefore providing an indication about 
the similarity between these two vectors. As the inner 
product is an intrinsic component of convolution, the sys¬ 
tematic application of neurons to signals along a given 
domain becomes related to convolution, being therefore 
related to convolutional artificial neuronal networks. 

After describing the simple and elegant principle of lin¬ 
ear morphing of patterns, this concept as well as the in¬ 
terpretation of neurons as template matchers were consid- 
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Figure 13: The self-organized map (20x20 cortex at the center of the 
figure) obtained with respect to instances of the 9x9 patterns 0, |, 
and □, containing three respective regions marked by distinct colors. 
The weights of some of the cortical neurons indicated by arrows are 
also shown surrounding the cortex, illustrating their similarity with 
variations of the trained patterns, which account for generalization 
and redundancy in during recognition. Observe that the sequence 
of weights that ones obtain while moving along the cortical cells 
(e.g. cyan arrow) can be understood as a kind of pattern morphing. 


ered while presenting self-organizing maps, more specifi¬ 
cally a basic Kohonen network, also illustrating that tra¬ 
jectories along the respective ‘cortex’ tend to be related 
to the morphing between the trained patterns. 
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